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Abstract 

This paper presents a new and ubiquitous framework for establishing achievability results in network information 
theory (NIT) problems. The framework is used to prove various new results. To express the main tool, consider a set 
of discrete memoryless correlated sources (DMCS). Assume that each source (except one, Z") is randomly binned 
at a finite rate. We find sufficient conditions on these rates such that the bin indices are nearly mutually independent 
of each other and of Z n . This is used in conjunction with the Slepian-Wolf (S-W) result to set up the framework. We 
begin by illustrating this method via examples from channel coding and rate-distortion (or covering problems). Next, 
we use the framework to prove a new result on the lossy transmission of a source over a broadcast channel. We also 
~f^ prove a new lower bound to a three receiver wiretap broadcast channel under a strong secrecy criterion. We observe 

that we can directly prove the strong notion of secrecy without resorting to the common techniques, e.g., the leftover 



O 
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hash lemma. We have also used our technique to solve the problem of two-node interactive channel simulation and 

en . 

^-^ i the problem of coordination via a relay. 

I. INTRODUCTION 



Random coding and random binning are widely used in the achievability proofs of the network information 

U ■ 

theory (NIT) problems. Random coding is a channel coding technique that allows one to prove the existence of a 

good codebook (which is a subset of the product set A"r" T , := Yii=i ^T)' while random binning is a source coding 

technique that partitions the product set into bins with desired properties. 

Existing achievability proofs for source coding and channel coding problems are mostly based on repeated use of 

these techniques. Moreover some connections between certain source coding and channel coding problems has been 

observed. Slepian and Wolf, in their seminal paper on the lossless source coding [2|, interpreted the achievability 

of the rate R — H(X\Y) for compressing the source X n at rate R to a destination with access to the source Y n , 

through a channel coding problem. In contrast, Csiszar and Korner, obtained an achievability proof for multiple 

access channels (MAC) through the distributed source coding problem of Slepian and Wolf [3|. In a recent work 

J4|, Renes and Renner showed the achievability of the channel capacity via a combination of Slepian-Wolf (S-W) 

coding and privacy amplification. The main theme in these works is that the set of sequences mapped to the same 
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index through S-W coding constitutes a good channel code, and hence we have a decomposition of the product 
set into the channel codebooks. Similarly our framework treats source coding and channel coding problems on the 
same footing. 

Consider the problem of sending a message M over the channel q(y\x). Traditional random coding considers an 
encoder X n (M,F) and a decoder M(Y n ,F) where F is a common randomness, independent of M, available to 
both the transmitter and receiver. R.v. F represents the random nature of codebook generation. Since the probability 
of error is the average of that over all realizations of F, one can find / such that X n (M, F = /) and M(Y n , F = /) 
form appropriate encoder and decoder. We depart from this by first generating n i.i.d. copies of X n and Y n , Then 
we take both F and M to be functions of X n such that F is nearly independent of M. Note that we still have 
the property that p(y n \x n ,F = /) = p(y n \x n ) and p(m\F = /) ta p(m) meaning that X n (M,F = /) and 
M(Y n ,F — /) are legitimate choices as stochastic encoder and decoder. We construct F and M as random 
partitions (bindings) of X n . The question then arises that under what conditions two random bin indices are 
independent (as in the case of F and M), and what is the sufficient condition for recovering X n from Y n and a 
bin index F. 

To answer these questions in a more general framework, in Section [Til we prove two main theorems on approx- 
imating the joint pmf of the bin indices in a distributed random binning. We study properties of random binning 
in two extreme regimes, namely, when the binning rates are low and high. In the first case, we observe that if 
the rates of a distributed random binning are sufficiently small, the bin indices are nearly jointly independent, 
uniformly distributed and independent of a non-binned source Z". This result generalizes the one for the channel 
intrinsic randomness ifTUl . The second case is the S-W region, which shows that if the rates of distributed binning 
are sufficiently large, the outputs of random binning are enough to recover the sources. 

We argue that not only the new framework has a simple structure (it uses only random binning), but it can solve 
some problems much easier than the traditional techniques; see ifTHl . |[T9"1 for two examples. These examples consider 
the problems of channel simulation and coordination. In ifTSl we find an exact computable characterization of a multi- 
round channel simulation problem for which only inner and outer bounds were known previously. In coordination 
problems we want to generate random variables whose joint distribution is close to the desired i.i.d. distribution 
in total variation distance. Traditional techniques (such as packing and covering lemmas) commonly address the 
probability of error events. This is not general enough to cover all of the total variation distance constraints that 
show up in the coordination problems. In such cases one has to come up with new proof techniques. One particular 
case is the resolvability lemma used by Cuff (see 1T31 Lemma 19]). 

Furthermore, we believe that the framework leads to more rigorous and simpler proofs for problems in secrecy. 
In problems of secrecy one has to deal with certain equivocation rates. Generally speaking there are two general 
techniques for proving lower bounds on equivocation rates: one is to prove existence of "good" codebooks with given 
properties. These good codebooks are then used to compute the equivocation rates. This approach was originally 
used by Csiszar and Korner in [5]. The second approach is to compute the expected value of equivocation rates over 
codebooks, and prove existence of a "good" codebook with large equivocation rate (in the same way that a codebook 
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with small probability of error is identified). Some existing works on secrecy follow the second approach in a non- 
rigorous way. Instead of defining a random variable for the random codebook and conditioning the equivocation 
rates by that random codebook, they use the unconditioned distribution to calculate the equivocation rates. The 
recent book by El Gamal and Kim HI uses the second approach in a rigorous way. However in some scenarios 
calculation of equivocation rates conditioned on the codebook r.v. can be involved. We believe that our framework 
leads to simpler proofs in such cases. 

Moreover we can prove strong secrecy results for multi terminal scenarios. One way to prove strong secrecy 
results is to use privacy amplification of Maurer and Wolf |6] . Another approach is to use resolvability techniques 
0, |E). To best of our knowledge resolvability techniques are not developed for multi terminal scenarios except 
for one work on MACs [11]. The latter result has been used in [|9l to prove strong secrecy results for two-way 
channels. Our approach resembles the resolvability techniques, and is able to deal with strong secrecy in general 
multi terminal scenarios. 

This paper is organized as follows: in Section [nl] we begin by demonstrating our approach for some primitive 
problems of NIT, i.e. channel coding and lossy source coding problems, before getting into our new results. 
Then we apply our approach to two complicated scenarios, i.e., lossy coding over broadcast channels and a three 
receiver wiretap broadcast channel under a strong secrecy constraint. In Section [IV] we discuss connections between 
our framework and the covering and packing lemmas in a multivariate setup. We show that the set of typical 
sequences can be decomposed into covering and packing codebooks. Finally in Section [V] we give a summary of 
our framework. 

Notation: In this paper, we use Xg to denote (X, : j £ S), p u A to denote the uniform distribution over the set A 
and p(x n ) to denote the the i.i.d. pmf Y[7=iP( x i)> un l ess otherwise stated. The total variation between two pmf's 
p and q on the same alphabet X , is defined by \\p(x) — g(ar) |] x := h Y^ x \p(x) — q(x)\. 

Remark 1 Similar to H13V in this work we frequently use the concept of random pmfs, which we denote by capital 
letters (e.g. Px)- For any countable set X let A x be the probability simplex for distributions on X. A random pmf 
Px is a probability distribution over A*. In other words, if we use il to denote the sample space, the mapping 
w£!)h) Px(x;lu) is a random variable for all x £ X such that Px{x]ui) > and ^2 x P\{x; ui) — 1 for all u). 
Thus, uj t— > Px(-]Ui) is a vector of random variables, which we denote by Px- We can definite Px.y on product 
set X x y in a similar way. We note that we can continue to use the law of total probability with random pmfs 
(e.g. to write Px(x) — X)„ Pxy{x, y) meaning that Px(x; uj) = ^2 Pxy{x, y; uj) for all uj) and the conditional 
probability pmfs (e.g. to write Py\x{v\ x ) — p /"'\ meaning that Py\x(v\x',uj) = X p y' v '^> for all uj). 

II. Output statistics of random binning 

Let (X[i : t], Z) be a DMCS distributed according to a joint pmf px {1 . T] , z on a countably infinite set Ili=i ^ x %■ 
A distributed random binning consists of a set of random mappings Bi : X\ l — > [1 : 2 nRi ], i £ [1 : T], in which Bi 
maps each sequence of X™ uniformly and independently to the set [1 : 2 nRi }. We denote the random variable Bt(X™) 
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by B t . A random distributed binning induces the following random pmf on the set A"r". T , x Z n x J^[ t=1 [1 : 2"- Rt ][] 

T 

P(xf 1:T] ,z n ,b [1:T] ) =p(x? 1:T] ,z n )l[l{B t (x?) = b t }. 

t=\ 

One can easily verify that (Bi, ■ ■ ■ , Bt) are uniformly distributed and mutually independent of Z n in the mean, 
that is 

T 

EP(z n ,b [1:T] ) = 2- n ^^^p(z n )=p(z n )l[pf 1:2nIh] (b t ). 

t=i 

The following theorem finds constraints on the rate-tuple (i?i,--- ,Rt), such that the preceding observation 
about the mean holds for almost any distributed binning. We will be using this theorem frequently in the proofs. 
A more general form of this theorem is provided and proved in the appendix. 

Theorem 1 If for each <S C [1 : T], the following constraint holds 

Y,Rt<H{X s \Z), (1) 

tes 



then as n goes to infinity, we have 



E 



P{z n ,b [1:T] )- P (z n )l[pf 1 :2n^ ] (bt) 



0. (2) 



Remark 2 In [10], the channel intrinsic randomness was defined "as the maximum random bit rate that can be 
extracted from a channel output independently of an input with known statistics". One can generalize this definition 
to the broadcast channel Pxu. T] \z> m the sense of finding random bit rates (R±, • • • , Rt) that can be extracted 
individually from the channel outputs independently of the channel input and the other random bits. Theorem\l\gives 
an achievable rate region for this scenario and implies that random binning is sufficient to prove the achiev ability^ 

Theorem [T] enables us to approximate the pmf P{z n ,b\-.T). We now consider another region for which we can 
approximate a specified pmf. This region is the Slepian-Wolf region for reconstructing X".^ in the presence of 
(Bi-t, Z n ) at the decoder. As in the achievability proof of the [12, Lemma 7.2.1], we can define a decoder with re- 
spect to any fixed distributed binning. We denote the decoder by the random conditional pmf P sw (x^.j,-, \z n , &[i : ri) 
(note that since the decoder is a function, this pmf takes only two values, and 1). Now we write the Slepian-Wolf 
theorem in the following equivalent form. 

Lemma 1 If for each S C [1 : T], the following constraint holds 

Y,Rt>H(X s \X Sc ,Z), (3) 

tes 

'We are using capital letter P on the left hand side since the pmf induced on xV" „, , z n , b[i : T] is random. 

2 In fact, 1 10 1 considered the case of a "general channel" and a "general input" (as defined in [121) and proved their results using the information 
spectrum methods. However, we observe that the general form of the theorem proved in the appendix generalizes to this setting and shows that 
one should substitute average entropy with the spectral inf-entropy, (121 to get the result for the general case. 
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then as n goes to infinity, we have 



E 



P(x™ 1:T j,2; n ,x™ 1:T ]) — p(x™ 1:T ], z™)l{X[ 1:T ] — a;™ 1:T ]} 



0. 



Proof: From the definition of the total variation, we know that \\p(x) — <z(aO||i = 12x-p(x)>q(x)ip( x ) ~ l( x )]- 
Using this property we can write 

(a) 



E 



P\ x [\:T]i z i x \V.T]) ~ P{ x [l:T]i z )^-\ x [l:T] ~ X [1:T]S — ^ 2_^ P( X [1:T] J z ™' ^[i:T]) 



[l-.T]' ' [i:TJ- 



= EP(Xp i:T] ^Xp i:T] )^0, (4) 

where (a) follows from the fact that whenever P(x[' 1 . T , , z™,:^.,™) > ^(a;^.,™, z n )l{x5. T , = x]\. T A we must have 
1{o;m. t , = 2;n. T i} = 0. To see this note that P(x? 1 . T ,, 2 n ,#R. T i) — p(x? 1 . T ^z n )P(x? l . T Ax? l . T , 1 z n ). ■ 

III. ACHIEVABILITY PROOF THROUGH PROBABILITY APPROXIMATION 

In this section, we illustrate the use of Theorem Q] and Slepian- Wolf binning (LemmaQ]) through some examples. 
Before going through these examples, we state two useful lemmas on total variation of arbitrary (random) pmfs. 

Definition 1 For any random pmfs Px and Qx on X, we say Px ~ Qx if E \\Px — Qx\\i < (■■ Similarly we use 
Px ~ Qx for two (non-random) pmfs to denote the total variation constraint \\px — <3x|li < e - 

Lemma 2 We have 

1) £/J Lemma 17]: \\pxPy\x ~ QxPy\x\\ 1 = \\px ~ qx\\i 

Sl3\ Lemma 16]: \\p x - qxWt < \\pxPy\x - Qxqy\x\\ r 

e . 2e 

2) If PxPy\x ~ QxQy\X' then there exists x £ X such that Py\x=x ~ Qy\x=x- 

2') More generally the probability of the set {x G X : Py\x=x ~ Qy\x=x} under both px and qx is at 
least 1 — 2y/e. 

3) If P x ~ Qx and P x Py\x ~ PxQy\x, then P x P Y \x ~ QxQy\x- 

Proof: The proof of the first part can be found in |fl3l . Next consider the second part. To prove this, we bound 
above the expectation E px ||_Py|x — Qy\x ||, as follows: 

e px \\py\x -gyixllj = ^2px(x) f -^2\p Y \x(y\x) - q Y \x(y\x)\ J 

x \ y / 

- 2^2\ Px ^ PY \ x ^ x ^ - 1x(x)q Y \x(y\ x )\ + 2 Yl \lx{x)q Y \ x (y\ x ) -Px( x )q Y \x(y\ x ) 

= \\pxPy\x - qxQ^x^ + \\qxqY\x -Pxq Y \x\\ x 
= \\pxpy\x -qxq Y \x\\ l + \\qx -px\\ x 

< 2||pxPr|x -^x^yixIIj 

< 2e, (5) 
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where in the steps (a) and (b) we use the first part of this lemma. Thus there exists a specified x <G X such that 2) 
hold. To show 2') we use Markov's inequality 

/ r , v ii ii /-,\ . e px ||Pr|x -9r|x||i ^ . / - 

v e 
Finally, consider the third part of the lemma. By the triangular inequality and the first part of the lemma, we 
have 

E \\PxPy\x - QxQy^W, < E \\PxPy\x - PxQy^W, + E \\P x Qy\x - QxQy^W, 

= E\\P x P Ylx - PxQy^W.+KWPx -Qxlli 
< e + <5. (6) 



Lemma 3 If d: X Y.y —} [0,d max ] is a distortion measure, pxy is a pmf with E pxY d(X, Y) = D and qxY is a 
pmf such that qxY ~ Pxy, then we have 

E qxY d{X,Y)<D + ed max . (7) 

Proof: We have 

E qxY d(X,Y) = ^2q X Y(x,y)d(x,y) 
x,y 

< ^2pxY(x,y)d(x,y) + ^2\q X Y(x,y) - p X Y{x,y)\d(x,y) 

x.y x.y 

< D + d max y^ \qxy{x,y) -pxy{x,y)\ 

<D + ed max (8) 



A. Channel coding 

Consider a channel with the conditional pmf p(y\x). We want to prove that for any p(x), R < I(X; Y) is an 
achievable rate for the channel. Let M. = [1 : 2 nR ] be the message set. The joint pmf of (M,X n ,Y n ) is equal 
to p'^ /l (m)p(x n \m)p(y n \x n ), where p(x n \m) can be non-deterministic (although it is deterministic in the standard 
definition) and M has uniform distribution due to the definition of channel coding. Here we want to determine 
p{x n \m) and a decoder such that we have reliable communication. In our approach, we let (X n ,Y n ) be i.i.d. and 
distributed according to p(x,y). We define two random mappings on X n as follows: to each x n , we assign two 
random bin indices m € [1 : 2 nR ] and / e [1 : 2 nR ], uniformly and independently. Then the induced random pmf 
is 

p(x n ,y n )P(m\x n )P(f\x n ) = P(m,f)P(x n \m,f)p(y n \x n ). 
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Now, Slepian- Wolf binning shows that if R > H(X\Y), the decoder can reliably reconstruct X n using (F,Y n ) 
and therefore decode M which is a function of X n . Thus, for almost any F = /, the decoding is reliable. Hence, 
if the transmitter and receiver agree on an appropriate F = /, then we can choose P(x n \m, /) as the encoder and 
the Slepian-Wolf decoder as a decoder. Note that P(y n \x n ,F = /) = p(y n \x n ), so conditioning on / simulates 
a DMC channel. However, notice that after conditioning on F = /, P(m\f) is not necessarily uniform. But if 
R + R < H(X), Theorem Q] implies that M has uniform distribution and roughly speaking, is independent of F. 
Hence R < I(X; Y) is achievable using this encoder and decoder. 

To set up a general proof structure that we will use consistently throughout this paper, we restate the above proof 
in the following framework: 

The proof is divided into three parts. In the first part we introduce two protocols each of which induces a pmf on a 
certain set of r.v.'s. The first protocol has a negligible probability of error but leads to no concrete coding algorithm. 
However the second protocol is suitable for construction of a code, with one exception: the second protocol is 
assisted with a common randomness that does not really exist in the model. In the second part we find constraints 
implying that the two induced distributions are almost identical. In the third part of the proof, we eliminate the 
common randomness given to the second protocol without disturbing the probability of error significantly. This 
makes the second protocol useful for code construction. 

Part (1) of the proof: Take some arbitrary p(x). We define two protocols each of which induces a joint distribution 
on random variables that are defined during the protocol. 

Protocol A. Let (X n , Y n ) be i.i.d. and distributed according to p(x, y) — p(x)p(y\x). 

Random Binning: Consider the following random binning: to each sequence x n , assign uniformly and indepen- 
dently two bin indices to E [1 : 2 nR ] and / G [1 : 2 nR ]. Further, we use a Slepian-Wolf decoder to recover x n 
from (y n , /). We denote the output of the decoder by x n . The rate constraint for the success of the decoder will 
be discussed later, although this decoder can be conceived even when there is no guarantee of success. 

The random |^| pmf induced by the random binning, denoted by P, can be expressed as follows: 

P(x n , y n , m, /, x n ) = p(x n ,y n )P(m, f\x n )P sw (x n \y n , f) 
= P{m, /, x n )p{y n \x n )P sw {x n \y n , f) 
= P{m, f)P(x n \m, f)p(y n \x n )P SW (x n \y n , /). (9) 

Protocol B. In this protocol we assume that the transmitter and the receiver have access to the shared randomness 
F where F is uniformly distributed over [1 : 2 n ]. Then, the protocol proceeds as follows: 

• The transmitter chooses a message to uniformly distributed over [1 : 2 nR ] and independently of F. 

• In the second stage, knowing (to, /), the transmitter generates a sequence x" according to the conditional pmf 
P(x n \m, f) of the protocol A. Then it sends x n over the channel. 

3 The pmf is random due to the random binning assignment in the protocol. 
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• At the final stage, the receiver, knowing (y™, /) uses the Slepian-Wolf decoder P sw (x n \y n , f) of protocol A 
to obtain an estimate of x n . 
The random pmf induced by the protocol, denoted by P, factors as 

P(x n ,y n ,m,f,x n ) = p u (f)p u (m)P(x n \m,f)p(y n \x n )P sw (x n \y n ,f) (10) 

Part (2a) of the proof: Sufficient conditions that make the induced pmfs approximately the same: To find the 
constraints that imply that the pmf P is close to the pmf P in total variation distance, we start with P and make 
it close to P in a few steps. The first step is to observe that m and / are the bin indices of x n in Protocol A. 
Theorem Q] implies that if R + R < H(X) then there exists e n —¥ such that P(m, /) t£ p u (m)p u (/) = P(m, /). 
Equations (O and (TTOb imply 

P(m,f,x n ,y n ,x n ) e &P(m,f,x n ,y n ,x n ) (11) 

Part (2b) of the proof: Sufficient conditions that make the Slepian-Wolf decoder succeed: The next step is to 
see that for the Slepian-Wolf decoder of protocol A to work well. Lemma Q] requires imposing the constraint 
R > H(X\Y). It yields that for some vanishing sequence S n , we have 

P(m, f, x n , y n , x n ) k P(m, f, x n , y n )l{x n = x n }, (12) 

Using equations (fTTT i. (fT2l and the third part of Lemma [2] we have 

P(m, f, x n , y n , x n ) £ "« " P(m, f, x n , y n )l{x n = x n }. (13) 

Part (3) of the proof: Eliminating the shared randomness In the protocol we assumed that the transmitter and 
the receiver have access to shared randomness F which is not present in the model. Nevertheless, we show that 
the transmitter and the receiver can agree on an instance / of F, Using Definition [T] equation dT3l guarantees the 
existence of a fixed binning with the corresponding pmf p such that if we replace P with p in ( TTOb and denote the 
resulting pmf with p, then p(m, f, x n , y n , x n ) "sa " p(m, f, x n , y n )l{x n = x n }. Now, the second part of Lemma 

2(e„+5„) 

[2] shows that there exists an instance / such that p(m,x n ,y n ,x n \f) ~ p(m, x n ,y n \f)l{x n = x n }. Using 

2(e„+5„) 

the second item in part 1 of Lemma|2]we conclude that p(x n ,x n \f) sw l{x n — x n } which is equivalent to 
p(X n t^ X n \f) < 2(e„ + S n ). Reliable recovery of the transmitted X n implies reliable recovery of the message 
M. 

Finally, identifying p(x n \m, f) as the encoder and p (x n \y n , f) as the decoder results in a pair of encoder- 
decoder with the probability of error at most 2(e„ + S n ). 

B. Lossy source coding 

Problem definition: Consider the problem of lossy compression of a source within a desired distortion. In this 
setting, there is an i.i.d. source X n distributed according to p(x), an (stochastic) encoder mapping X n to M S [1 : 
2 nR ], a decoder that reconstructs a lossy version of X n (namely Y n ) and a distortion measure d : X xy -^ [0, d max \- 
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A rate R is said to be achievable at the distortion D, if E{d(X n ,Y n )) < D + e n , where e n ->• and d(X n ,Y n ) 
is the average per letter distortion. 

Statement: Here we wish to reprove the known result on the achievability of the rate R > I(X;Y) for any 
p(x, y) where Ed(X, Y) < D. 

Overview of the proof: Note that the joint pmf between (X n ,M,Y n ) factors as p(x n )p(m\x n )p(y n \m). The goal 
is to determine p{m\x n ) and p(y n \m) as the pair of encoder-decoder. As the case of channel coding, let (X n , Y n ) 
be i.i.d. and distributed according to p(x,y). We have E p ^ n ^ n - ) d(X n ,Y n ) — Ed(X,Y) < D. Again, to each y n , 
we assign two random bins m and / at the rates R and R, respectively. Since the decoder aims to produce Y n 
which has the desired distortion with X n , we assume that R+R > H(Y) resulting in reliable decoding of Y n from 
(M,F), Hence, we expect that if the encoder and decoder agree on an appropriate value of F = /, the decoder 
can reliably recover Y n using the S-W decoder. However, conditioning on F = f changes the distribution p(x n ) 
to P(x n \f) and we obtain an scheme that achieves distortion D w.r.t. P(x n \f) rather than p(x n ). Despite this fact, 
roughly speaking, Theorem [TJ says that F and X n are independent and P(x n \f) is close to p(x n ) if R < H(Y\X). 
Hence, we can choose the encoder-decoder w.r.t. P(x n \f) for p(x n ). 

Proof: Take some arbitrary p(x, y) where Ed(X, Y) < D. The proof is again divided into three parts. In the first 
part we introduce two protocols each of which induces a pmf on a certain set of r.v.'s. The first protocol has the 
desired distortion property but leads to no concrete coding algorithm. However the second protocol is suitable for 
construction of a code, with one exception: the second protocol is assisted with a common randomness that does 
not really exist in the model. In the second part we find constraints implying that the two induced distributions are 
almost identical. In the third part of the proof, we eliminate the common randomness given to the second protocol 
without disturbing the distortion significantly. This makes the second protocol useful for code construction. 

Part (1) of the proof: We define two protocols each of which induces a joint distribution on random variables 
that are defined during the protocol. 

Protocol A. Let (X n ,Y n ) be i.i.d. and distributed according to p(x,y). 

Random Binning: Consider the following random binning: to each sequence y n , assign uniformly and indepen- 
dently two bin indices m € [1 : 2 nR ] and / e [1 : 2 nR ]. Further, we use a Slepian-Wolf decoder to recover y n 
from (m, /). We denote the output of the decoder by y n . The rate constraint for the success of the decoder will 
be discussed later, although this decoder can be conceived even when there is no guarantee of success. 

The random pmf induced by the random binning, denoted by P, can be expressed as follows: 

P(x n ,y n , m, /, y n ) = p(x n , y n )P{f\y n )P{m\y n )P sw {y n \m, /) 

= P(f,x n ,y n )P(m\y n )P SW (y n \mJ) 

= P(f, x n )P(y n \x n J)P(m\y n )P sw (y n \m, f). (14) 

Protocol B. In this protocol we assume that the transmitter and the receiver have access to the shared randomness 
F where F is uniformly distributed over [1 : 2 ]. Then, the protocol proceeds as follows: 
• The transmitter generates Y n according to the conditional pmf P(y n \x n , f) of protocol A. 
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• Next, knowing y n , the transmitter sends m which is the bin index of y n . Random variable M is generated 
according to the conditional pmf P(m\y n ) of protocol A. 

• At the final stage, the receiver, knowing (m, /) uses the Slepian-Wolf decoder P sw (y n \m, /) of protocol A 
to obtain an estimate of y n . 

The random pmf induced by the protocol, denoted by P, factors as 

P(x n ,y n , m, /, y n ) = p U (f)p(x n )P(y n K, f)P(m\y n )P sw \y n \m, f) (15) 

Part (2a) of the proof: Sufficient conditions that make the induced pmfs approximately the same: To find the 
constraints that imply that the pmf P is close to the pmf P in total variation distance, we start with P and make it 
close to P in a few steps. The first step is to observe that / is a bin index of y n in protocol A. Theorem [T] implies 
that if R< H(Y\X) then there exists e„ -)■ such that P(f,x n ) rs P U (f)p(x n ) = P(f,x n ). Equations (Q4) and 
(fTBT l imply 

P(m, f, x n ,y n , y n ) « P(m, /, x n ,y n , y n ) (16) 

Part (2b) of the proof: Sufficient conditions that make the Slepian-Wolf decoder succeed: The next step is to 
see that for the Slepian-Wolf decoder of protocol A to work well. Lemma Q] requires imposing the constraint 
R + R > H(Y). It yields that for some vanishing sequence S n , we have 

P(m, f, x n , y n , y n ) » P(rn, f, x n , y n )l{y n - y n }. (17) 

Using equations ([Tol l, ( ITTt and the third part of Lemma |2] we have 

P(m,f,x n ,y n ,y n ) e "«" P(m,f,x n ,y n )l{y n = y"}. (18) 

farf (5) o/ f/ie proof: Eliminating the shared randomness: In the protocol we assumed that the transmitter and 
the receiver have access to shared randomness F which is not present in the model. Nevertheless, we show that the 
transmitter and the receiver can agree on an instance / of F, Using Definition Q] equation ( fT8l guarantees existence 
of a fixed binning with the corresponding pmf p such that if we replace P with p in (TT~5T > and denote the resulting 
pmf with p, then p(m, f, x n ,y n , y n ) "sa " p(m, f, x n , y n )l{y n = y n } := p(m, f, x n ,y n , y n ). Using the second 
item of part one of Lemma|2] we have p(x n ,y n ) "» " p(x n ,y n ) — Px n Y n (x n , y n )- 

Applying lemma[3]to p x „y„ and px™Y n and noting that M pxnYn r x n^n\d(X n ) Y n ) < D, we obtain 

E §{x n ;0n) d(X n ,Y n )<D, 

for sufficiently large n. Using the law of iterated expectation, we conclude that there exists an F = f such that 
E p{xn ^ lf) d(X n ,Y n )<D. 

Finally, specifying p(m\x n , f) as the encoder (which is equivalent to generating a random sequence y n according 
to p(y n \x n , f) and then transmitting the bin index m assigned to y n ) and p (y n \m, /) as the decoder results in 
a pair of encoder-decoder obeying the desired distortion. 
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C. Lossy coding over broadcast channels 

Problem definition: Consider the problem of lossy transmission of an i.i.d. source S n distributed according to 
p(s), over the broadcast channel p(yi, y 2 \x). Here, the sender wishes to communicate the source to the two receivers 
within desired distortions (D 1: D 2 ). Formally, there are 

• an encoder that assigns a random sequence x n to each s n according to p enc (x n \s n ), 

• two decoders, where decoder j = 1,2 assigns an estimate s™ € Sj to each y ri according to p j (s^ly 1 ?), 

• two distortion measures dj(s,§j). 

A distortion pair (Di,D 2 ) is said to be achievable, if there exists a sequence of encoder-decoder such that 
Ed 3 (S n , §?) < Dj + e„, j = 1, 2 and e„ -> 0. 
We now state a new result on the above problem: 

Theorem 2 A distortion pair {D\,D 2 ) is achievable for the lossy transmission of the source S over the broadcast 
channel p{yiy 2 \x), if there exist a pmf p(u\o :2 ]), an encoding function x(u[q :2 ],s) and two decoding functions 
§i(uQ,Ui,yi) and s 2 (uo,u 2 ,y 2 ) such that ~SLdj(S, Sj) < Dj, j = 1,2 and the following inequalities hold: 

I(U U j ;S)<I(U U j ;Y j ) ,j = l,2, 

I{U [ o..2 ] ;S) + I(U 1 ;U 2 \Uo)<mm{I(U ;Y 1 ),I{Uo;Y 2 )} 

+ I(U 1 ;Y 1 \U )+I(U 2 ;Y 2 \U ), 

I(U Uii S) + I(U U 2 ; S) < I(U Ui; Y x ) + I(U U 2 -Y 2 ) 

- I(Uv,U2\U S). (19) 

Remark 3 The above result is related to the result of Han and Costa, H16V , H14J for the lossless transmission of 
correlated sources over broadcast channels when S is of the form (Si,S 2 ). In this case we can include Si in U\, 
and S 2 in U 2 . If we take the distortion function to be the hamming distance function, the above bound reduces 
to a weaker version of the result of Han and Costa since instead of a vanishing probability of error we have a 
vanishing distortion. However the proof can be modified to recover the result of M4V . 

Proof: Take some arbitrary p(s,U[o :2 i,yi,y 2 ) and functions x(uiq :2 us), s 1 (-u ,'«i,2/i) an d s 2 (u ,u 2 ,y 2 ) such 
that Kdj(S, Sj) < Dj, j — 1,2. The proof is again divided into three parts. In the first part we introduce two 
protocols each of which induces a pmf on a certain set of r.v.'s. The first protocol has the desired distortion 
properties but leads to no concrete coding algorithm. However the second protocol is suitable for construction of a 
code, with one exception: the second protocol is assisted with a common randomness that does not really exist in 
the model. In the second part we find constraints implying that the two induced distributions are almost identical. 
In the third part of the proof, we eliminate the common randomness given to the second protocol without disturbing 
the distortion significantly. This makes the second protocol useful for code construction. 

Part (I) of the proof: We define two protocols each of which induces a joint distribution on random variables 
that are defined during the protocol. 
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Protocol A. Let (S n , U%. 2 i, V™, Y 2 ) be i.i.d. and distributed according to p(s, Um-w, J/i, 2/2)- 
Random Binning: Consider the following random binning: 

• To each sequence 1$ assign uniformly and independently a random bin index fo € [1 : 2 nR °], 

• For j = 1, 2, to each pair (uq , w") assign uniformly and independently a random bin index fj <G [1 : 2™^], 

• We use a Slepian-Wolf decoder to recover u r l 1: u™ from (y\, fo, fi), and another Slepian-Wolf decoder to 
recover u^ 2 ,u 2 from (yj , /o, /b)- Note that we denote the two estimates of Uq by the two receivers with «q j^ 
and Mq 2 - The rate constraints for the success of these decoders will be imposed later, although these decoders 
can be conceived even when there is no guarantee of success. 

• Random variable s™ is created as a function of (v,q , u", y™) and .§2 i s created as function of (uq , «2 > 2/2 ) usm g 
the two decoding functions given at the beginning. 

The random pmf induced by the random binning, denoted by P, can be expressed as follows: 

P(* n ,uf O :2].^0M[O:2],fiS,l,fi?,^ 

xP^(^2,«2ly2\/0,/2) 

= P(/[0:2], *", «rO:2])P(l/"l:2]l«rO:2]. '"l^Kl^l ltf> /o, A) 

xP SH/ (^ 2 ^2ly 2 ",/o,/ 2 ) 

= P(/[0:2],* n )P(«ro : 2]l/[0:2],« n )p(l/Pl:2]l«rO:2],* n )P SW (^,l,flll»l l ,/o,/l) 

xP SH/ (^ 2 ,«2|y2",/0,/ 2 ). (20) 

We have ignored s" and Sj from the pmf at this stage since they are functions of other random variables. They 
will be introduced later. 

Protocol B. In this protocol we assume that the transmitter and the two receivers have access to the shared 
randomness F[ 0:2 ] where F[ 0:2 ] is uniformly distributed over [1 : 2 nR °] x [1 : 2 nRl ] x [1 : 2 nR2 }. Observe that this 
implies that Fo, F\ and F^ are mutually independent. Then, the protocol proceeds as follows: 

• The transmitter generates ?7S. 2 i according to the conditional pmf P(u? Q . 2 i\s n , /[o:2]) of protocol A. 

• Next, X n is computed from (U£. 2 i, S n ) using n copies of the function x(u[o-.2], s) (the arbitrary function we 
chose at the beginning). R.v. X n is transmitted over the broadcast channel. 

• At the final stage, the receiver j = 1,2, knowing (y™, fo, fj) uses the Slepian-Wolf decoder P sw (ii l ■ , Uj\y 7 j', fo, fj) 
of protocol A to obtain estimates of Uq and w". We note that while the receiver j — 1,2 knows fo, f\ and fi, 

it uses only fo, fj in its Slepian-Wolf decoder. 

• Random variable s" is created as a function of (wo i>^i>2/™) anc ' *2 ^ s created as function of (u l 2 ,u 2 ,y 2 ) 
using the two decoding functions given at the beginning. 

The random pmf induced by the protocol, denoted by P, factors as 

P(S n , Up. 2 ] , Vli J/2 > /[0:2] ! ^0,11^1' ^0,2 1 ^2 ) = 

P^/lO^MO^*]!^/^^ (21) 
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Again we have ignored s" and s 2 from the pmf at this stage since they are functions of other random variables. 

Part (2a) of the proof: Sufficient conditions that make the induced pmfs approximately the same: To find the 
constraints that imply that the pmf P is close to the pmf P in total variation distance, we start with P and make it 
close to P in a few steps. The first step is to observe that in protocol A, /q is a bin index of Uq, /i is a bin index 
of (uq,u") and fi is a bin index of (uq,u 2 ). Substituting X\ — Uo,X2 = UqUj,Z = S in Theorem Q] implies 
that S n is nearly independent of F[o-.2] an d that its pmf is close to p(s n ), if 

Ro < H(U \S), 
Ro + RjKHiUoUjlS) ,j = l,2, 

R Q + Ri+R 2 <H(U [0:2] \S). (22) 

In other words, the above constraints imply that there exists e„ — > such that P(f[o-.2]i s n ) & p u (f[o-.2])p(s n ) = 
P(f [0:2], $")■ Equations d20l > and ( f2Tb imply 

P(s n ,ufo,2 ] ,yi,y2JlO-.2 ] ,ul^U , l,ul2,U2) « ?(»", ^0:2] , Vl, «2. /[0:2], ^0,1, $1, ^0,2, $?) ( 2 3) 

farf (2/7) of the proof: Sufficient conditions that make the Slepian-Wolf decoder succeed: The next step is to see 
that for the Slepian-Wolf decoder of protocol A to work well. Lemma Q] for X\ = Uq, X 2 = UqUj, Z = Yj yields 
that the decoding of UqU" is reliable if, 

R + R j >H(JJ V j \Y j ), 

Rj > H(Uj\UoYj) for j = 1,2. (24) 

It yields that for some vanishing sequence S n , we have 

P(s n , uf . 2] , vl, y%, f [Q . 2] , u™ 4 , u™, u™ 2 , u") « P(s n , uf 0:2] , yl, yl, / [0 . a] ) 

x l{^o,i = «o.2 = «o, 6" = «i . <S = «a>- (25) 

Using equations (f23), (f25) and the third part of Lemma [2] we have 

P{s n ,u^ 2] ,vl, yj, / [0:2] , u " 4 , «?, < 2 , u 2 l ) £ "« " P(*», U f 0:2] , yf, |tf, / [0:2] ) 

x l{uj tl = < 2 = ug, 6? = u?, ^ = u£}. (26) 

Part (3) of the proof: Eliminating the shared randomness: In the protocol we assumed that the transmitter and 
the two receivers have access to shared randomness Fip-,2] which is not present in the model. Nevertheless, we show 
that the transmitter and the receivers can agree on an instance /r 0: 2] of F[o-.2]- Using Definition Q] equation d26l ) 
guarantees existence of a fixed binning with the corresponding pmf p such that if we replace P with p in d2"TT > and 
denote the resulting pmf with p, then 

p(s n , Uf 0:2] ,yl, l/J, f [0 :2],U0,li «1 1 "0,2. «?) '' « " ?>( s "> "p*]' *# ' y 2 . /[0:2]) 

x l{«g; 1 = «£ 2 = < , 7i" = u'l, V% = M 2 } 

: = !>(*"» «p:2],l/l . 2/2 , /[0:2],«5,1, «1 , <2, «?)■ 
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Using part one of lemma [2] we can introduce s\ and $2 in the above equation. Random variable s" was a function 

of (mq 1; u™,y™) and s 2 was function of (u^ 2 ,U2 , 2/2 )■ 

P( S > U [0:2]>yi>y2 J[0:2],U ,1' U 1' U 0,2 I U 2> S 1 I S 2) ~ -P( S . "[0:2] . ft . !/2 J[0:2] J 

xp(*f !«?,!, «?,»rMS5|<a,«?,vr) 

:= P(s n ,uf . 2] ,2/™, 2/2 , f[0:2],UQ } l,u",Uo }2 , u 7 2 \ s?, Ig). 

Note that because of the indicator function terms in p, p(S n , S", §2) is an i.i.d. marginal distribution according 
to the pmf that we started with at the beginning. Thus, under the probability measure p the distortion constraints 
Edj(S n ,S™) < Dj, j = 1,2 are satisfied. Using the first part of lemma [2] we can drop all the random variables 
except s n , s™ , s 2 l to get: 

pKJ^g^Sy,^,^ (27) 

Remember that the pmf p corresponded to Protocol B which was appropriate for coding . 

Unlike the previous case of lossy source coding where we used the law of iterated expectation at this stage, we 
need to use a concentration result since we are dealing with two distortion functions. Since p(S n , S", S$ ) is an 
i.i.d. distribution we can use the weak law of large number (WLLN) to get that 

dj{S n , S?) -> E § dj(S n , §?) < Dj, in p gn§n . 

Thus there exists e n — > such that 

p{(« n ,S?,*S) : dj(8 n ,%) < Dj ,j = 1,2} > 1- e n . 

Using equation (|27"l i, the probability of the same set with respect to p should be converging to one, that is 

p{(s n , SJ, S2 1 ) : dj(s n , «?) < Dj , j = 1, 2} > 1 - e„ - e„ - <5„. 

Thus, there exists some /ro : 2] such that 

p({(« n , s", 15) : d,(* B , J") < £>,- , j = 1, 2}|^ [0:2] = / [0:2] ) > 1 - e n - e n - 6 n . 

This would imply that 

%=»*»„ , WS n , 4™)]) < 0j + (?n + e n + S n )d max . 

where we have used the fact that the distortion functions are bounded. 

Specifying (p(u" :2] |/ [n:2] ,s n ),x"(up 0:2] ,s™)) as the encoder and (p SW (&o,j , Uj\Vj , fo,fj),&j(uo,j,Uj,y")) as 
the decoder j results in encoder and decoders obeying the desired distortions. 

Finally applying Fourier-Motzkin elimination (FME) on d24b and d22"b gives jl% . ■ 
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D. Wiretap broadcast channels with strong secrecy criterion 

Problem definition: Consider the problem of secure transmission over a broadcast channel with a wiretapper, 
p(yi,V2,z\x). Here, we wish to securely transmit a common message too G [1 : 2 nR °] to the receivers Yi,Y 2 and 
two private messages rrij 6 [1 : 2 nRj ],j = 1,2 to the receivers Yj,j — 1,2, respectively, while concealing them 
from the wiretapper. We use the total variation distance as a measure for analyzing the secrecy. Formally speaking 
there are, 

• Three messages Mq,Mi,Mz which are mutually independent and uniformly distributed, 

• A stochastic encoder p enc (x n \rri[o :2 ]), 

• Two decoders, where decoder j assigns an estimate (moj,rhj) of (too,TOj) to each y ri . 

A rate-tuple (i? , #1,-^2) is said to be achievable if Pr{Uj = i i2 (-Moj, Mj) 7^ (M ,Mj)} — > and M[ 0:2 ] is 
nearly independent of the wiretapper output, Z n , that is, 



P(m [0 :2] , Z n ) ~ Pm [0:2] ( m [0:2] )p(z T 

where, here p(z n ) is the induced pmf on Z n and is not an i.i.d. pmf. 



0. 



Remark 4 The above total variation constraint is stronger than weak secrecy [10]. It is equivalent with strong 
secrecy when the above total variation distance converges to zero exponentially fast, we can modify our tools 
(Theorem\l\and Lemma\l} for finite alphabets to have an exponential decrease. However we are dealing with both 
finite and countable alphabet sets in this work. 

Theorem 3 A rate-tuple (i?o, Ri, R2) is achievable for the secure transmission over the wiretap broadcast channel, 
if it belongs to the convex hull of 

Ro + Rj < I(U Uj;Yi\Q) - I(UoUj;Z\Q), j = 1, 2. 
R + R ± +R 2 < min {I(U ; Y t \Q), I(U ; Y 2 \Q)} + I(Ui\ Yx\U Q , Q) 

+ I(U 2 ;Y 2 \U ,Q)-I(U 1 ;U 2 \U Q ,Q)-I(U m ;Z\Q) 
2R Q + R 1 + R 2 < HUoUuYilQ) - I(U Ui; Z\Q) + I(U U 2 ;Y 2 \Q) 

- I(U Q U 2 ; Z\Q) - I(U V , U 2 \U , Z, Q) (28) 

where Q, Um.w — X — (Yi, Y 2 , Z) forms a Markov chain. 

Proof: For simplicity we consider the case where the time-sharing r.v. Q is a constant random variable. One 
can incorporate this into our proof by generating its i.i.d. copies, and sharing it among all parties and conditioning 
everything on it. 

Take some arbitrary p(u[ 0:2 ], x)p(yi, y 2 , z\x). The proof is again divided into three parts. In the first part we 
introduce two protocols each of which induces a pmf on a certain set of r.v.'s. The first protocol has the desired 
reliability and security properties but leads to no concrete coding algorithm. However the second protocol is suitable 
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for construction of a code, with one exception: the second protocol is assisted with a common randomness that does 
not really exist in the model. In the second part we find constraints implying that the two induced distributions are 
almost identical. In the third part of the proof, we eliminate the common randomness given to the second protocol 
without disturbing the distortion significantly. This makes the second protocol useful for code construction. 

Part (1) of the proof: We define two protocols each of which induces a joint distribution on random variables 
that are defined during the protocol. 

Protocol A. Let (UI^. 2 ,,X n , Y™, Y 2 n , Z n ) be i.i.d. and distributed according to p(u\o : 2],x,yi,y2,z). 

Random Binning: Consider the following random binning: 

• To each Uq assign uniformly and independently two random bin indices mo € [1 : 2"^°] and /o € [1 : 2 nR °], 

• For j = 1, 2, to each pair (uq, u") assign uniformly and independently two random bin indicies rrij G [1 : 2 nRj ] 
and fj e [1 : 2 nk i). 

• We use a Slepian-Wolf decoder to recover u r l i,Wi from (j/™,/o,/i), and another Slepian-Wolf decoder to 
recover Mq 2 , u 2 from (y 2 , /o, / 2 ). Note that we denote the two estimates of Uq by the two receivers with Uq x 
and Uq 2 . The rate constraints for the success of these decoders will be imposed later, although these decoders 
can be conceived even when there is no guarantee of success. 

The random pmf induced by the random binning, denoted by P, can be expressed as follows: 

P{ TL TL TL Tl P A Tl A Tl •" 71 A TL\ I Tl TL TL Tb\ T~t / C \ Tl \ T~t / I Tl \ 

(«[0:2] .3/1.2/2.* . ™[0:2] , J [0:2] , U ,l. «1 . M 0,2. u 2 ) = P{U[0:2] > Vl . 2/2 . * ) P (/[0:2] |u [0: 2] )-P("l[0:2] |w [0:2 ]) 

X P^o^UlW, /0, /l)P SW K 2 , ^2 l?/2 , /0, /2) 

= P(/ [0: 2],TO [0: 2],Mf 0:2] Ma;"|uf 0:2] )p(?/i,2/ 2 l ,2™|a;"') 

x P SH/ « 1; fiflyr, /o, fi)P SW {ul 2 , u n 2 \y^, /o, / 2 ) 
= P(/[0:2],m [0:2] )P(u[ 0:2] |/ [0:2] ,m [0:2] )p(x n |up 0:2] )p(y™,y™,z n |a;") 

xp^^^^i^./o./op^^.^i^./o.^). 

(29) 
Protocol B. In this protocol we assume that the transmitter, the two receivers and the wiretapper have access to 
the shared randomness P[o :2 ] where P[o :2 ] is uniformly distributed over [1 : 2"^°] x [1 : 2 nRl ] x [1 : 2 nR2 ]. Observe 
that this implies that Fq, F\ and P 2 are mutually independent. Then, the protocol proceeds as follows: 

• The messages Mo, M\ and M 2 are mutually independent of each other and of P[o :2 ], uniformly distributed 
over [1 : 2"^°] x [1 : 2 nRl ] x [1 : 2 nR2 ]. 

• The transmitter generates U^. 2 -< according to the conditional pmf P(uf . 2 ,|m[o : 2]) /[0:2]) of protocol A. 

• Next, X n is generated according to the n i.i.d. copies of the conditional pmf p(x|u[ 0:2 ]) (computed from the 
arbitrary p(x, M[0:2l) we chose at the beginning). R.v. X n is transmitted over the broadcast channel. 

• At the final stage, the receiver j = 1,2, knowing (y™, /o, fj) uses the Slepian-Wolf decoder P sw (Uq ■, u 1 f\y 1 j, /o, fj) 
of protocol A to obtain estimates of Uq and u". We note that while the receiver j — 1,2 knows /o, f\ and / 2 , 

it uses only /o, /j in its Slepian-Wolf decoder. 
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• We use the output of the S-W decoder j = 1,2 for decoding of the messages (Mq,Mj), In protocol A, we 
constructed Mo as a bin index of Uq . Here the receiver j has an estimate of Uq which is denoted by Uq . To 
find the estimate of Mo, the decoder can apply the same indexing pattern used in the random binning of Uq 
to the estimate Uq*. In other words, we want to use the same Pm \U" °f protocol A. Thus, decoder j passes 
Uq, through the conditional pmf Pm |£/"(^ 1 o,j|wo ,-) to get the bin index assigned to Uqa. Similarly, we had 
constructed the message Mj as a bin index of (Uq, U™). To construct the estimate rhj, we use the conditional 
pmf P Mi \u$,u™ (mj\v%j,v%) of protocol A. 
The random pmf induced by the protocol, denoted by P, factors as 

-P( U [0:2] ' 2/l\ 2/2 ! z ™> TO [0:2] j /[0:2] ) ^0,1' "l ' ^0,2; ^2 ) 

= P U (f[ :2] )P U (m [0:2] )P(uf 0:2] |m [0:2] ,/ [0:2] )p(x n \uf 0:2] )p(yf 1:2] , z n \x n )P sw (<% tl ,i%\y?J Ji)P sw (uZ 2 ,i%\y2,f ,f2). 

(30) 

We have ignored M-r.v.S from the pmf at this stage since they are (random) functions of other random variables. 
Part (2a) of the proof: Sufficient conditions that make the induced pmfs approximately the same: To find the 
constraints that imply that the pmf P is close to the pmf P in total variation distance, we start with P and make it 
close to P in a few steps. The first step is to observe that in protocol A, (m , Jq) is a bin index of Uq, (mi, /i) is 
a bin index of (uq, u") and (ra 2 , f<i) is a bin index of (uq, u 2 ). Substituting X\ = Uq,X 2 = UqUj, Z = constant 
in Theorem Q] implies that Mm.™ is nearly independent of Fm. 2 ] if 

Rq + Rq<H(U ), 
Rq + Rj +R + Rj < H(UqU 3 ) ,j = l, 2, 
R a + Ri + R 2 + Ro + Ri+R2< H(U [0:2] ). (31) 

In other words, the above constraints imply that there exists e„ — > such that P(f [o:2]> m [o-.2]) ^ P U (f[Q:2])p U ( m [n:2]) — 
-P(/[0:2]i TO [0:2])- Equations (|29l l and ( f30l > imply 

P ( M [0:2] . Vl> 2/2 > «"> ™[0:2], /[0:2], "o,l' "l ' "0,2l &2 ) ^ f> ( U |o:2]> 2/") 2/2 ) A m [0:2] , /[0:2] , «0,1> "l > &0,2> «2 )■ ( 32 ) 

Part (2b) of the proof: Sufficient conditions that make the Slepian-Wolf decoder succeed: The next step is to see 
that for the Slepian-Wolf decoder of protocol A to work well. Lemma Q] for X\ = Uq, X 2 = UqUj, Z = Yj yields 
that the decoding of UqUj 1 is reliable if, 

Ro + Rj >H(U U j \Y j ), 

Rj > H(Uj\U Yj) for j = 1,2. (33) 

It yields that for some vanishing sequence 5 n , we have 

P ( U lO:2],yi'y2, zn , m iO:2]JiO:2],Uo t i,Ui,ul 2 ,U 2 ) « P(uf Q . 2] , y? , y% , Z™ , m [0:2 ] , / [0: 2] ) 

x I{u5 (1 = u£ a = uj, u» = <, ^ = <}. (34) 
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Using equations ( |32l ), ( f34b and the third part of Lemma [2] we have 

£(«[0:2] > 2/™' Vi > »", n»[ .a], /[0:2],«0,1, «1, «0,2. «!l) £ ' « " P («fo:2] . Vl > 1/2 . 2 "> m [0:2] , /[0:2]) 

X IK,! = < 2 = uff,fi? = <,«2 = «?}• (35) 

Using first part of Lemma [2] we have 

T~% / 77 77 /* *"• 77 ^ 77 rt 77 ^77\ H^ ^ti. ~-^ / 77 77 /* \ 

P(u [0:2] ,z ,m [0:2] ,/ [0:2] ,u 01 ,u 1 ,u 02 ,u 2 ) « F(m [0:2] ,z , m [0:2] , / [0:2] ) 

x l{^,i = fij, 2 = «?, fi? = <, 62 = «£}• (36) 

Part (2c) of the proof: Sufficient conditions that make the protocols secure: We must take care of independence of 
M[o : 2], and (Z™, i*[o:2l) consisting of the the wiretapper's output and the shared randomness. Consider the random 
variables of protocol A. Substituting X\ = Uq,X2 = UqUj,Z = Z in Theorem Q] implies that Aij 0: 2] is nearly 
independent of (Z n , F[o:2l) if 

Ro + R <H{U \Z), 

Ro + Rj + R + Rj < H(U Uj\Z) ,j = 1, 2, 

Ro + Ri + R 2 + Ro + Ri + R 2 < H(U m \Z). (37) 

In other words, the above constraints imply that there exists e„ — > such that 

P(z n ,f [0-.2], m [0:2] ) e £ P (z n )p u (f [0:2] )p u (m [0:2] ). (38) 

Also observe that the pmf P(z n ) is equal to i.i.d. pmf p(z n ) in Protocol A. 
Using equations ( 136b and ( 1381 and the third part of Lemma [2] we have 

P(uf Q:2] , « n ,m l o ! 2],/[0:2].«0 1 l.«l. «0,2. «!?) e " + W + £ " p(z n )p C/ (/ [0:2] )p C/ (m [0:2] )P( M f 0:2] |z™, m [0:2 ],/[0:2]) 

x l{«» fl = ^ 2 = «», «» = «»,«» = «»}. (39) 

fart (5) <?/ f/ze proof: Eliminating the shared randomness: In the protocol we assumed that the transmitter, the 
receivers and the wiretapper have access to shared randomness F[ . 2 ] which is not present in the model. Nevertheless, 
we show that the transmitter and the receivers can agree on an instance /[o :2 ] of Fro^i. Using Definition Q] equation 
(|39l guarantees existence of a fixed binning with the corresponding pmf p such that if we replace P with p in (l30b 
and denote the resulting pmf with p, then 

P( u [0:2]> A TO[0:2],/[0:2],«0,1) «1) "0,2) «2 ) ^ K2™)^(/[0:2])p y (ro[0:2]M u [0:2] I 2 ™) m [0:2] , /[0:2]) 

X 1{U™ ! = U™ 2 = «J, U? = <, U™ = W^}. 

Now, the second part of Lemma [2] shows that there exists an instance /[o :2 ] such that 

2(e„+<5„+e„) f/ 

P( u fo:2]: z " 1 m [0:2],/[0:2],'"o ! i,Ui,'Uo !2 ,U 2 |/ [0:2] ) W p(«")P ("l[0:2])p(W[0:2] p"i m [0:2], /[0:2]) 

x 1{«J (1 = < 2 = t#, fi? = u?, «J - «£}. 
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Reliability: Using the second item in part 1 of Lemma [2] we conclude that 



which is equivalent to p ((UqjU™) ^ (C/q 1 , C^")|/[o:2] J < 2(e„ + <5„). Reliable recovery of the transmitted 
(Uq, C/") implies reliable recovery of the message (Mo, Mj) in receiver j. This is because AIq is a bin index 
of Uq and Mj is a bin index of (Uq, <7") in protocol A. We have used the same conditional pmf in protocol 
B. After de-randomizing the pmf in the above paragraph, M will become a deterministic function of Uq. A 
similar statement holds for Mi and M 2 . 

i—i 2(e„+<5„+e„) 

• Security: Using the second item in part 1 of Lemma |2] we conclude that p(z n ,m[Q :2 ]\f[Q :2 \) ~ 

P U (m [0: 2])p(z n ) 
Finally, identifying p(x n \m,[Q :2 ] , f[o-.2]) (which is done via generating U[q :2 ] first) as the encoder and the Slepian-Wolf 
decoders results in reliable and secure encoder-decoders. 

Applying FME on ( BTT ), d33l ) and d37] i gives | |28) . Note that the equations of ( BTT i are completely redundant. ■ 

Remark 5 If one only wants to securely transmit a common message Mq as in HI 51/ (i.e. i?i = R 2 = 0), the 
second inequality of the (1241) can be neglected, because it is sufficient to recover Mq through only Uq; it is not 
necessary to make sure that we do not make any error in decoding [7™ and U 2 . This resembles the idea of indirect 
decoding of N air and El Gamal. Applying FME gives the following lower bound on Rq which subsumes the lower 
bound given in [15] under weak secrecy criterion: 

R = max min \l(U U V , Yi\Q) - I(U U V , Z\Q),I(U U 2 ; Y 2 \Q) - I(U U 2 ; Z\Q), 

PQU [0 . 2] x I 

i(/(J7 17i; Y X \Q) - I{U U i; Z\Q) + I(U U 2 ; Y 2 \Q) - I(U U 2 ; Z\Q) - I{U X ; U 2 \Q, U , Z)) } 

We now compare our lower bound with the one given in [15 J. The lower bound given in [15] is the maximum of 

min{/(17oE7i;yi|Q) - I(U U 1 ;Z\Q),I(U U 2 ;Y 2 \Q) - I(U U a ; Z\Q))} 

over all p(q, u )p(ui,u 2 , x\u )p(yi, y 2 , z\x) where I(U\,U 2 ; Z\Uq) < I(Ux; Z\Uq) + I{U 2 ; Z\U ) - I(Ui; U 2 \U ). 
We first note that because of the Markov chain Q — Uq — UiU 2 XZYiY 2 the above constraint is equivalent with 
I(U\, U 2 ; Z\Q, Uq) < I(U\, Z\Q, Uq) + I(U 2 ; Z\Q, Uq) — I(Ui; U 2 \Q, Uq). Algebraic manipulation shows that this 
constraint holds only when I(U\; U 2 \Q, Uq, Z) = 0. We note that the Markov constraint Q — Uq — U\U 2 X can be 
dropped. This is because given any (Q, Uq,U\, U 2 ,X) where the Markov chain does not hold we can replace Uq 
with (Uq, Q) and find a new set of random variables where the Markov chain holds and the inner bound expression 
remains unchanged. To sum this up, the lower bound of [15] can be rewritten as the maximum of 

mia{l(U U 1 ;Y l \Q)-I(U U 1 ;Z\Q),I(U U 2 ;Y 2 \Q)-I(U U 2 ;Z\Q))^ 

over all p(q, Uq, U\, u 2 , x) where I(U\, U 2 \Q, Uq, Z) = 0. It is clear that when I(U\, U 2 \Q, Uq, Z) = our lower 
bound reduces to this lower bound. In general it may be higher because we are taking the maximum over all 
p(q, it[o:2], x) without any constraint. 
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IV. Covering and Packing: Revisited 

Most of the achievability proofs in NIT are based on two primitive lemmas, namely packing lemma and covering 
lemma [1|. Thus it would be interesting to see how our probabilistic proofs relate to these lemmas. We show that 
Theorem 1 implies a certain form of multivariate covering (but not exactly the one mentioned in [1|). The discussion 
on packing lemma is similar and hence omitted. 

Multivariate covering: We prove a version of multivariate covering that is similar to Marton coding JT]. Consider 
r.v.'s Xr 1: T]Z. Roughly speaking, we want to prove that under certain conditions on Ri's, there exists a partition 
of set of typical sequences of Af" into 2 nRi bins of size 2 nR * = 2 n ^ H ^ Xi ^~ Ri ' for i = 1 : T, such that if we 
choose any of the partitions of X[ L , and any of the partitions of Xg, etc, we can find sequences x™, x^,---, xVj, in 
these partitions such that they are jointly typical with each other and with Z n with high probability, for almost 
all choice of partitions. The conditions imposed on the rate of the bins, R[ are given in inequality (l40b . This is a 
generalization of the mutual information terms showing up in Marton coding and match the ones reported in 0]. 

To show this let T™[X\i-mZ\ be the set of strongly typical sequences w.r.t. px {1 . T] z- Theorem Q] says that if 

V<S C [1 : T] : ^R' t > £ H(X t ) - H{X S \Z), (40) 

tes *g5 

then -P(fo[i : T]j z n ) ~ P U (bi 1 .T])p(z n ). One can show that with high probability the number of the typical sequences 
assigned to each bin hi € [1 : 2 nRi ] is about 2 nRi , for i = 1 : T, provided that i?j < H(Xi) (for example, 
through the same lines as in the proof of balanced coloring lemma in lfT7l ). This fact alongside with Theorem Q] 
implies that there exists a fixed binning with the corresponding pmf p such that p(z n ,b[ 1 . T ]) rj p u (b[ 1 . T ])p(z n ) 
and the number of the typical sequences assigned to each bin hi e [1 : 2 nRi ] is about 2 nRi , provided that (l40b 
is satisfied. Let q(b[i : x], xl\. T ,, z n ) — p u (b[i :T ])p(z n )p(xV 1 . T Ab, z n ). Since p(x? 1 . T p z n ) — p(xV 1 . T ,, z n ), we have 
piT^lX^.^Z] ) < e n ->• 0. Markov inequality and q w p imply that qB [1:T] ({b[i-.T] '■ QflT[X[i:T\Z] c \b[i:T\) > 
y/e^}) — > 0. Therefore for almost all the choices of b\x-T\, the probability of the typical set conditioned on 6[i : t] 
is large, implying a non-zero intersection of the typical set and the product partition set. 

V. Summary of the framework 

We are ready to state the steps of the probabilistic proof, precisely. The probabilistic proof is done in three steps: 

1) Specifying r.v.'s via which the information is transmitted, e.g., X in the channel coding, C/o : 2 in the last two 
examples. 

2) Solving the S-W problem corresponding to decoding the desired information at each receiver through random 
binning, e.g., decoding of E/S-ii m tne ^ ast two exam pl es at decoder 1. 

3) Sharing an appropriate bin between terminals such that decoding is reliable, while ensuring that conditioning on 
a bin does not affect the problem formulation, e.g. the uniformity of the message in channel coding and independence 
of the messages and wiretapper's output in the secrecy problems, etc. 
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Appendix A 
Proof of TheoremQ] 

We prove a slightly generalized version of Theorem Q~]in this section. To do this, we need some more definitions 
most of which are taken from |[T2l . 

Definition 2 ( 11121 . Sec. 1.3) A general source X is a sequence of sequences of random variables, defined as 
follows: 

x = {x n = (x[ n \x^,...,xt ) )}°° 

where X n is a random variable over the n-th Cartesian product X n of an arbitrary alphabet X with respect to 
an arbitrary probability distribution px™- The distribution px n is not necessarily Ltd. nor does it even need to 
satisfy the consistency condition as defined in H12V . We restrict ourselves to general sources with countable X. A 
set of correlated general sources (X, Y, • • • ) is defined similarly. 
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Definition 3 The spectral sub-entropy rate of X is defined as 

-ff(X) = p — lim inf — log ■ 



n->oo n px"(X n ) 

Similarly for any two correlated general sources X and Y, the conditional spectral sub-entropy rate of X given 
Y is defined as 

#(X|Y) =p-liminf-log 



oo n ° p xn \ Yn {X n \Y n ) 

Remark 6 For the case of discrete memoryless correlated sources the conditional spectral sub-entropy rate reduces 
to the usual conditional entropy by the WLLN. 

Definition 4 A distributed random binning of correlated general sources Xfi-j-i, Z consists of a sequence of a set 
of random mappings B™ : Xj 1 — > [1 : 2 nRi ], i G [1 : T], in which of 1 maps each sequence of X" uniformly 
and independently to the set [1 : 2 nRi ]. We denote the random variable B t n (X") by B\ f 1 . A random distributed 
binning induces the following random pmf on the set X\[.j<\ x Z n x J3 t=1 [l : 2 nRt ], 

T 

P(xf 1:T] ,Z n ,b [1:T] )= PX n T] , Z n(xl T] ,Z n )l[l{Bi n \x^ = b t }. 

t=l 

One can easily verify that (_B 1? • • • , Bt) are uniformly distributed and mutually independent of Z n in the mean, 
that is 

T 

EP(z n ,b ll:T] ) = 2- n ^ R *p zn (z n )=p Z n(z n )llpf 1:2nHt] (b t ). 



Theorem 4 If for each S C [1 : T], the following constraint holds 

Y,Rt<K(Xs\Z), 
then as n goes to infinity, we have 



tes 



E 



P(z n ,b [llT] )- PZ n(z n )l[pf 1:2nRt] (b t 
t = \ 

Proof: We define the following typical set, 

K ■= {(X[i-.T],z n ) ■ {x n s ,z n ) € A:(S),VS C [1 : T]} . 
where -4™(«S) is defined as follows: 

A:(S) := |(xS,z") : -^logp X n ]Z n(x n s \z n ) > H(X S \Z) - e, J . 
Since for each S C [1 : T], lim n _j. 00 p(*4"( l S)) = 1, we have lim n _ >00 p(^l") = 1. Next note that 

T 

P(z n ,b [1:T] ) = ]T p{x? 1:Ti ,z n )'[[l{B<r\x?) = b t }. 



(41) 



(42) 



(43) 



h [l:T] 



t-1 
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We can decompose this sum into two parts (similar to 0~3)) as follows: 

T 

P(z n ,b [UT] ) = Y p(xf 1:Tl ,z n )l[l{Bi n \xt) = b t } 

[1:TJ- L — 1 



23 



P(z n ,b [UT] ) = J2 p(xl 1:Tl ,z n )l[l{B\ r\x?) = b t } 



*[1:IT 



t = \ 



We then have 



Thus 



P(z n , b [1:T] ) = P(z n , b [1:T] ) + P(z n , 6 [1:T] ). 



EP(z n , b [1:T] ) = EP(z n , b [1:T] ) + EP(z n , b [1:T] ). 
Using triangle inequality we have 



^'\ W ~ PZ" (^" ) n ^:2"^) 



E||P(«",6 [1: t|)-EP(« b ,6 [1: t|)|| 1 



< E 



P(z n ,b [1:T] )-Ep(z n ,b [1 .. T] ) 

P(z n ,b [1 .. T] )-EP(z n ,b [1 .. Tl 



(44) 



Therefore we need to show that both of the terms on the right hand side converge to zero as n converges to infinity. 
For the second term we have 



E 



P(z n ,b lUT] )-EP(z n ,b [UT] )\\ i <^ J2 2Ep (z n >h 

z n <b[i,T] 



T]l 



E Yl P(zVf 1:T]l 6 [1:T] ) 

E Y P(z n ,X[l:T]) 

Z > X [1:T]'{ Z i X [l:T]'Y- e 

Px~z~ ((A?) c ) -> 0, 



(45) 



where in the first step we use the inequality E\X — EX\ < 2E\X\ and in the last step, we use the relation 
P(z n ,x? t . T ,) — px n z n {z n ,xV' l . T ^). Next consider the first term of the r.h.s. of equation (04). Using Cauchy- 
Schwarz inequality we have 



Y E\P(z n ,b [m )-EP(z n ,b [1 .. T] )\< Y ^™{P{z^b {1 .. T] )). 

Z n ,b[-L:T] 2",6[1:T] 



(46) 
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Applying the formula \far(Y,J =1 X t ) = Z)i<i,j<T COv(Xi,Xj) gives 
var(P(z",& [1:T] )) - £ cov (p( x fi:T]^ n n{B^ T] (xf 1:T] ) = 6 [1:T] } 



[1:T]' [1:T]' 



,p(x[ l 1:T] ,z")l{^ ] (Sf 1:T] ) = 6 [1:T] } 
XI p(sf 1:T] ,z")p(xf 1:T] ,^) 



[1:T]' [1:T]' 



?(«) C„n \ _ J, \ iroW 



cov (1 {BW ](a; « :r] ) = 6[1:T]}; l{6^(j» :]1 ) = 6[1:T] }j (47) 



To evaluate the covariance term, we first partition the set {{xV 1 . t ^^xV 1 . t t) : (xV 1 . T ^z n ) £ A™, (xm. t ,,z") € .A™} 
into the sets Af(S, z n ) defined below: 

Af(S,z n ) := {(xf 1:T] ,xf 1:T] ) :xg = icg, X? ^ £?, Vi e 5 C , 

(x" 1:T ],z™) € A™, (x" 1:T ],z™) €*4.™j. 

We note that for each pair (ijj.™,!! 1 ^) inside the set A/"(0, z"), the random variables l{Bf"L(xK. T i) = 6[i : t]} 
and l{B^"L(xK. r i) = 6[i : t]} are independent, thus the covariance between them is zero . Next for S ^ 0, we 
bound above the covariance for the pair (jcfx-Tl'^ri-Tl) e ■N'(S,z n ) as follows, 

?(") ^« ^ _ h \ i;hW <^™ ^ — ». x^\ «=— ic i;r(") ^« ■> — «( n ) 



cov (l{B^](xf 1:T] ) = 6 [1:T] }, l{^,(Sf 1:T] ) = 6 [1:T] }J < E l{B^> T] {x^ T] ) = B™ ,(xf 1:T] ) = 6 [l!t] } 

= E l{Bg\x%) = bs,Bg>(x£ B ) = B^(xf 1:T] ) = fe 5 c} 

_ 2 -» l (fi s +2i?so)^ ^ 

where in the last step, we use the fact that the random variables l{Bg (Xg) — bs}, l{B s n J(xg c ) = b$c} and 
l{BgJ (Xgc) = 65c} are mutually independent. 
Substituting (08]) in d4Vj gives 

var(P( 2 «,6 [1:T] ))< X J] p(xf 1:T] ,z"Mxf 1:T] ,z")2-^+ 2 ^) 

0#sc[i : T](x» :T] ,x» :T] )eAr(s, z «) 

<p 2 (z") ^ 2 ~ n( - Rs+2R ^ Y P 2 ( x s\ zn ) 



X 



Y p{x%\x n s ,z n )p{x n Sa \x n S: z n ) 



= p 2 (z n ) Y 2-^ Rs+2R ^ Y P 2 (xs\z n ) 

0#SC[1:T] z™:(zg,z™)e.4y(,S) 

<p 2 {z n ) V 2^ n(Rs+2Rsc+ — {XslY ^ e) (49) 

0#SC[1:T] 



where the last step follows from the definition of the set A™(S), 
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Substituting J49l ) in J46] l and using the inequality \JYl'i=\ a i ^ XT=i v^ yield 



J2 yJ™(P{z»,b [UT \))< E $>(z")2-*^i y >-^- e > 

z",6[l:T] 5C[1:T] z n 

< V^ 2 _ * ( ^ (Xs|y) "' Rs ~ <:) (50) 

5C[1:T] 

Finally if for each S C [1 : T] we have ife < H(X,s\Z) then ( TSOb vanishes as n goes to infinity. This concludes 
the proof. ■ 

Remark 7 If the correlated sources are i.i.d and have finite alphabets, one can use the set of strong typical 
sequence instead of the set A™ in the proof. Since the probability of the set ofnon- strong typical sequences vanishes 
exponentially in n, one can observe that the total variation distance in Theorem Q] also vanishes exponentially in 
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